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EXECUTIVE SUMMARY 


This report summarizes the Aviation Impact Variable (AIV) Editor 
evaluation conducted at the Aviation Weather Center (AWC) in 
Kansas City, Missouri, by ACT-320, from February 13 through 23, 
1996. Specific results, conclusions, and recommendations for the 
evaluation are detailed within this report. 

Under agreement with the Federal Aviation Administration (FAA), 
AWC issues Area Forecasts, Airmen Meteorological Statements 
(AIRMETs), Significant Meteorological Statements (SIGMETs), 
Convective SIGMETs, and International SIGMETs for aviation use. 

To accomplish these tasks, AWC forecasters access gridded weather 
data produced by forecast models running at the National Centers 
for Environmental Protection in Camp Springs, Maryland. In order 
to produce graphical or textual aviation weather products AWC 
forecasters manipulate or "edit" the gridded weather data to 
modify model output. The National Oceanic and Atmospheric 
Administration (NOAA) Forecast Systems Laboratory (FSL), Boulder, 
CO, has been tasked with the development of automated 
productivity tools to enable forecasters to perform this data 
editing in a rapid and efficient manner. 

In response, FSL has developed the AIV Editor for viewing and 
editing of,weather data grids. Forecasters can use the editor to 
manipulate translational algorithm parameters as.well as editing 
individual grid points. The current version of the editor 
addresses in-flight icing potential only. 

A joint evaluation of the AIV Editor was conducted by the FAA 
William J. Hughes Technical Center and FSL. The evaluation took 
place in the Experimental Forecast Facility (EFF) forecaster work 
area mockup at AWC. The mockup simulates the current forecaster 
work areas. The objectives of the AIV Editor evaluation were to: 
(1) assess the human/computer interface, (2) determine the 
operational utility of the editing and viewing concepts, and (3) 
to determine whether or not the editing and viewing concepts 
conform to job task and information requirements of AWC 
forecasters. 

Many users indicated that training was inadequate; therefore, 
they were uncomfortable using several editor functions, 
especially the higher level concepts (i.e.. Vertical 
Interpolation, Algorithmic Parameters, and Volume Viewer). The 
effect of training was evident in the results, as the higher 
level concepts received less than acceptable utility and ease of 
use ratings while most other concepts received acceptable 
ratings. However, almost all require some improvement. 
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The conclusions and recommendations contained within this report 
should be assessed for their feasibility and integrated into the 
AIV Editor, if possible. While these recommendations will not 
resolve every problem, they will, if implemented overcome many 
problems currently experienced by the forecaster. 
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1. INTRODUCTION. 


The National Centers for Environmental Prediction (NCEP) Aviation 
Weather Center (AWC) located in Kansas City, Missouri, issues 
Area Forecasts, Airmen Meteorological Statements (AIRMETs), 
Significant Meteorological Statements (SIGMETs), Convective 
SIGMETs, and International SIGMETs for aviation use. AWC 
forecasters issue these products for the contiguous United States 
and portions of the Atlantic and Pacific Oceans. 

In order to produce the various aviation products, AWC 
forecasters access gridded weather data produced by forecast 
models running at NCEP in Camp Springs, Maryland. Based on 
additional meteorological information, the AWC forecasters may 
have the need to manipulate or "edit" the gridded weather data in 
order to produce more accurate graphical or textual aviation 
weather products. The National Oceanic and Atmospheric 
Administration (NOAA) Forecast Systems Laboratory (FSL), Boulder, 
Colorado, has been tasked with the development of automated 
productivity tools to enable forecasters to perform this data 
editing in a rapid and efficient manner. 

In response,, FSL has developed the Aviation Impact Variable (AIV) 
Editor for viewing and editing of weather data grids. 

Forecasters can use the editor to manipulate translational 
algorithm parameters as well as editing individual grid points. 
The current version of the editor addresses in-flight icing 
potential only. 

Although not ready for operational use, the editing and viewing 
concepts of the editor are ready for evaluation to determine, 
their operational utility. Therefore, a joint evaluation of the 
AIV Editor was conducted by the Federal Aviation Administration 
(FAA) William J. Hughes Technical Center and FSL. The evaluation 
addressed the editing and viewing concepts used by the editor. 
Although the evaluation only addressed the editing of icing 
grids, the results of the evaluation can be considered applicable 
to the editing of other AIVs such as turbulence and clouds. 

The AIV Editor is currently in the Experimental Forecast Facility 
(EFF) at the AWC. The purpose of the EFF is to develop and • 
evaluate new forecasting techniques. The EFF has been configured 
into a mockup of the AWC forecaster operational work area. 

1.1 PURPOSE . 

The purpose of this evaluation report is to document results of 
the AIV Editor evaluation. 
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1.2 SCOPE. 


This report svtmmarizes the AIV Editor evaluation conducted at 
AWC. The report is written using FAA-STD-024B as a guideline. 
Section and paragraph titles were changed where appropriate. The 
contents of this report include evaluation descriptions, 
evaluation results, conclusions, and recommendations for future 
iterations of the AIV Editor. 


2. REFERENCE DOCUMENTS . 
FAA-STD-024B 

ESD-TR-86-278 

MIL-STD-1472D 

AFOTEC Pamphlet 99-102 

3. SYSTEM DESCRIPTION. 


Federal Aviation Administration 
Standard Content and Format 
Requirements for the Preparation of 
Test and Evaluation Documentation, 
August 22, 1994. 

Guidelines for Designing User 
Interface Software, August 1986. 

Military Standard 1472-D, Human 
Engineering Guidelines, March 14, 
1989. 

Air Force Operational Test and 
Evaluation Center Software 
Usability Evaluation Guide, Volume 
4, June 1994. 


3.1 MISSION REVIEW. 


Not Applicable. 

3.2 SYSTEM CONFIGURATION . 

The AIV Editor is a software application currently configured to 
operate on a Hewlett-Packard HP755 workstation and running within 
the commercially available Application Visualization System 
(AVS). NCEP gridded weather data is provided to the AIV Editor 
via FSL. The data is provided as test data for demonstration and 
evaluation purposes. 

3.3 SYSTEM OVERVIEW . 

AIV Editor products include Icing Potential based on a National 
Center of Atmospheric Research/Research Applications Program 
(NCAR/RAP) algorithm, satellite imagery, and graphic plots of 
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Pilot Reports (PIREPs). The NCAR/RAP algorithm diagnoses Icing 
Potential by using temperature and relative humidity criteria 
applied to gridded output from the NCEP Rapid Update Cycle (RUC) 
model. The Icing Potential can be displayed as a composite, by 
individual flight levels, or a subset of flight levels. 
Additionally, categories (e.g., stable, unstable) of the Icing 
Potential algorithm can be displayed. Interactive display 
functions include zoom, pan, and reset. PIREPs and satellite 
images are time-matched to the Icing Potential. 

3.3.1 Viewing Concepts . 

The AIV Editor utilizes four different viewing windows: 

a. Horizontal Cross-Section, 

b. Vertical Cross-Section, 

c. Three-Dimensional Volume Viewer, and 

d. Point Soupdings of RUC temperature, dew point, and wind 
plotted on a Skew-T log P diagram. 

3.3.1.1 Horizontal Cross-Section . 

The Horizontal Cross-Section is the primary window for viewing 
and manipulating data. Icing Potential, satellite imagery, and 
icing PIREPs are displayed and each can be individually toggled 
on or off as desired. A spatial loop of the Icing Potential in 
the Horizontal Cross-Section can be created and viewed. 

3.3.1.2 Vertical Cross-Section . 

The Vertical Cross-Section enables the user to view a vertical 
slice of the Icing Potential from 1000 to 45,000 feet above mean 
sea level. The Vertical Cross-Section can be positioned by 
placing the mouse cursor at any point in the Horizontal Cross- 
Section window and clicking the mouse once. A white index line 
(oriented east-west or north-south according to user preference) 
in the Horizontal Cross-Section indicates the location of the 
Vertical Cross-Section. In addition, the user cari scan from 
north to south or from east to west in the Vertical Cross- 
Section. ' 

3.3.1.3 Three-Dimensional Volume Viewer . 

When selected, the Three-Dimensional (3-D) Voltme Viewer overlays 
the Horizontal Cross-Section. This provides the user with a 3-D 
image of the Icing Potential. Selected flight levels are . 
highlighted in red. The Icing Potential can be viewed from a top 
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down, east, west, north, or south view. The user can also enter 
an arbitrary azimuth view from 0° to 360°. Additionally, the 
user can manually rotate the image using the mouse. 

3.3.1.4 Point Sounding . 

The Point Soundings window displays the vertical RUC soundings of 
temperature, dewpoint, and winds. The sounding point can be 
selected by using the mouse to select a point on the grid or by 
selecting a site from the location identifiers menu. 

3.3.2 Editing Condepts . 

Data can be edited via two methods: 

a. Graphical Editing, and 

b. Interactive Algorithm Editing. 

With either method, the user can edit all or parts of the grid. 

3.3.2.1 Graphical Editing . 

Graphical Editing is performed in the Horizontal Cross-Section. 
During graphical editing, a user can choose to either add or 
delete Icing Potential from a specified area for select flight 
levels. An undo button allows the user to remove any unwanted 
edits. Additionally, by using the Vertical Interpolation 
feature, a user can add an area of Icing Potential through a 
sloping layer. 

3.3.2.2 Interactive Algorithm Editing . 

Interactive Algorithm Editing enables the user to modify the 
Icing Potential by adjusting algorithm parameter thresholds. The 
thresholds are modified utilizing slider bars on the display. 

3.3.3 Edited Grids . 

After editing, the user has the option of saving the edited grids 
for future use. Since the AIV Editor is a prototype, no 
connections to existing AWC systems existed for the distribution 
of the edited grids. 

4. EVALUATION DESCRIPTION . 

4.1 SCHEDULE AND LOCATION .■ 

Questionnaires for the evaluation were distributed to forecasters 
at AWC on February 13, 1996. Questionnaires were returned by 
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February 23, 1996. The remainder of the evaluation was conducted 
at AWC from February 20 through 23, 1996. 

4.2 PARTICIPANTS . 

Personnel from the following organizations conducted and 
supported the AIV Editor evaluation: 

Organization Role 

ACT-320 Test Director and Evaluators 

FSL Technical Support, Observers, 

and Test Equipment 

AWC Forecasters and Test Equipment 

4.3 SPECIALIZED EVALUATION EQUIPMENT . 

The evaluation took place in the EFF forecaster work area mockup. 
The mockup simulates the current forecaster work area with the 
exception of the Automation of Field Operations and Services 
(AFOS) equipment. The AFOS equipment is being phased out of use 
at AWC. In place of the AFOS equipment, an extra Automated 
Weather Interactive Processing System (AWIPS) display developed 
for National Centers was installed. 

4.4 EVALUATION OBJECTIVES AND CRITERIA . 

4.4.1 Objectives . 

The objectives of the AIV Editor evaluation were: 

a. Assess the human/computer interface; 

b. Determine the operational utility of the editing and 
viewing concepts; and 

c. Determine whether or not the editing and viewing 
concepts conform to job tasks and information requirements of the 
AWC forecasters. This objective does not pertain to system speed 
or processing performance. 

4.4.2 Criteria . 

FSL was not required to design to formal system specifications; 
therefore, no formal success criteria were used for the 
evaluation. However, in order to identify both positive and 
negative attributes during the evaluation, the following success 
criteria were used: 
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a. Mode user ratings of less than 3 on the 5-point Likert 
scale for editing, viewing, and user-system interface concepts 
(1 = completely acceptable, 5 = completely unacceptable) and 
favorable user comments; and 

b. General agreement with human engineering design 
guidelines. 

4.5 EVALUATION DESCRIPTION . 

The evaluation program was structured to identify problem areas 
with product/function utility, editor interface, data 
presentation, and job task and information requirements of the 
editor. Results and collected data will be provided to FSL for 
use in further editing and viewing development. 

The FAA Technical Center, Communication/Navigation/Surveillance 
Engineering and Test Division, Weather Branch (ACT-320), in 
conjunction with FSL, developed procedures and tools for the AIV 
Editor evaluation. 

Four technic[ues were utilized during the evaluation to identify 
problem areas with the editor. The four techniques were: 

a. Questionnaire administration, 

b. Scenarios, 

c. Open forum, and 

d. Human Factors design guidelines comparison. 

4.5.1 Questionnaire Administration . 

The objective of the questionnaire was to identify problem areas 
perceived by the AWC forecasters, assess utility and ease of use, 
and solicit suggestions for improvement. The areas assessed were 
AIV Editor viewing, editing, and user interface concepts. 
Questionnaire results were also compared to operational utility 
as derived from the scenarios. 

4.5.2 Scenarios . 

The objective of the scenario was to lead a forecaster in 
providing systematic and organized feedback. The scenario 
directed forecasters to assess and edit, if necessary, the icing 
grids in a specified geographical region for a specific valid 
time. Icing areas and valid times differed based on actual 
weather conditions at the time of scenario completion. In 
addition, evaluators asked structured questions to further 
facilitate information collection and clarify forecaster 
comments. 


6 



4.5.3 Open Forijm . 


The objective of the open forum was to provide a medium for 
additional comments and questions. Forecaster comments and 
questions were noted and summarized. 

4.5.4 Human Factors Design Guidelines Comparison . 

The objective of comparing the AIV Editor interface to the human 
factors standards and guidelines was to ensure that good human 
factors design principles are used in future iterations of the 
editor. It is important to note that these standards and 
guidelines were not used for pass/fail type criteria, as the AIV 
Editor was not designed to specifically meet these guidelines. 

4.6 DATA COLLECTION AND ANALYSIS METHOD . 

4.6.1 Questionnaire Analysis . 

Descriptive statistics were used to summarize the data. Given 
the variability of the data, the mean was not appropriate. 
Therefore, the mode was the most appropriate statistic. The mode 
represents the rating most frequently selected by users. In some 
cases, a concept received a bimodal rating. A bimodal rating 
indicates two ratings were equally occurring, suggesting that 
users' opinions were split. In addition to the descriptive 
statistics, questionnaire comments were summarized. 

Questionnaire results are presented in section 5.1.1. Raw 
questionnaire data is presented in appendix A. 

4.6.2 Scenario Analysis . 

Forecaster procedures and comments from the scenario sessions 
were summarized and are presented in this report. A complete 
summary of user comments is presented in appendix B. 

4.6.3 Human Factors Guidelines Analysis . 

Results from the comparison of the human factors standards and 
guidelines to the AIV Editor display were summarized. Results 
were tabulated using a binary method of met or not met. These 
results are summarized in sections 5.3.1, 5.3.2, and 5.3.3. ' 

5. RESULTS AND DISCUSSION . 

5.1 QUESTIONNAIRE RESULTS . 

This section presents the results of the questionnaire. Section 

5.1.1 discusses the analysis of the rating scale while section 


7 



5.1.2 contains a summary of user comments concerning editing and 
viewing concepts. 

5.1.1 Rating Scale Results . 

Table 1 contains the mode utility and ease of use ratings for the 
viewing concepts. All of the viewing concepts received an 
acceptable rating with the exception of the volume viewer. The 
volume viewer received a borderline rating in both the utility 
and ease of use categories indicating that this concept requires 
significant improvement before it can be considered useful by 
forecasters. 

TABLE 1. MODE UTILITY AND EASE OF USE RATINGS FOR VIEWING 
CONCEPTS 


Viewing Concept 

Utility ■{n*l4j 

'iase of.Uee 
{n*14) 

Data File Selector 

2* 

• 2 

Display Options 

2 

2 

Horizontal Cross-Section 

2 

2 

Vertical Cross-Section 

2 

2 

Loop Controls 

2 

2 

Flight Level Selectors 

2 

2 

Data Options 

2 

2 

Skew-T 

2 

2 

Satellite Imagery 

2 

2 

Volume viewer 

3** 

3 

viewing Angles 

2 

2 


l=Completely Acceptable, 2=Acceptable, 3=Borderline, 
4=Unacceptable, 5=Completely Unacceptable 


NOTE: 

* n=13 for data file selector concept 

** Volume Viewer received a bimodal rating of 2 and 3. The 
rating of 3 was selected given that the distribution curve was 
more heavily skewed towards the higher end of the scale. 


Table 2 contains the mode utility and ease of use ratings for the 
editing concepts. All of the editing concepts received an 
acceptable rating with the exception of the vertical 
interpolation. The vertical interpolation concept received a 
borderline rating in both the utility and ease^ of use categories 
indicating that this concept requires significant improvement 
before it can be considered useful by forecasters. 
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TABLE 2. MODE UTILITY AND EASE OF USE RATINGS FOR EDITING 
CONCEPTS 


Editing Concept 

Utility |(n.=*l4) 

Ease of Use 
<n”14). 

Define Region 

2 

2 

Selecting Vertical Levels 

2 

2 

Delete Ice 

2 

2 

Add Ice 

2 

2 

Vertical Interpolation 

3 

3 

Algorithm Parameters 

2 

2 

Enable Flow 

2 

2 

Default Values 

1 

2 


l=Completely Acceptable, 2=Acceptable, 3=Borderline, 
4=Unacceptable, 5=Completely Unacceptable 


Table 3 contains the mode utility and ease of use ratings for the 
editor functions. All of the editor functions received an 
acceptable rating with the exception of the ease of use of the 
zoom function. Zoom ease of use received a borderline rating 
indicating that it would have to be improved before it can be 
considered useful by forecasters. 

TABLE 3. MODE UTILITY AND EASE OF USE RATINGS PER FUNCTION 


Function 

.^Utility 

Ease of. use 

Zoom 

2 

3 

Pan 

2* 

2 

Reset 

2 

2 

Undo 

2 

2 


l=Completely Acceptable, 2=Acceptable, 3=Borderline, 
4=Unacceptable, 5=Completely Unacceptable 


NOTE: 

* Pan received a bimodal rating of 2 and 3. The rating of 2 was 
selected given that the distribution curve was more heavily 
skewed towards the lower end of the rating scale. 


Response time and display readability were also rated. Response 
time received a borderline rating indicating that response time 
of the system would have to be improved before it can be 
considered useful by forecasters. Display readability received 
an acceptable rating. 
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5.1.2 User Comments. 


User comments from the questionnaire are discussed in the 
sections below. Comments are summarized for each editing and 
viewing concept as well as editor functions. 

5.1.2.1 General Comments . 

General user comments included increasing the image size, 
increasing font size or at least making font size user 
configurable, the addition of more PIREP information, looping of 
satellite imagery, addition of station identifiers, and the 
addition of icing intensities. 

5.1.2.2 Specific Comments . 

5.1.2.2.1 Data File Selector . 

Users assigned the Data File Selector a mode rating of 2; 
however, improvements were noted that would make the concept more 
useful. For instance, users would like to have data 
automatically updated as it is received. Users would like the 
selector to indicate the actual valid time, instead of the 
current indication of model run time plus forecast interval. 

■ 5.1.2.2.2 Display Options . 

The Display Options or Icing Category Selector received a mode 
rating of 2. User comments indicated that the concept would be 
more useful if icing intensity was displayed. 

5.1.2.2.3 Horizontal Cross-Section. 


The Horizontal Cross-Section was assigned a mode rating of 2; 
however, users suggested several improvements. Forecasters 
indicated they would like to see a larger data window. Also, the 
blue grids representing areas without, icing tended to degrade the 
satellite imagery. 

5.1.2.2.4 Vertical Cross-Section . 

The Vertical Cross-Section received a mode rating of 2; however, 
users suggested several improvements. Many users noted that 
PIREPs should be displayed in this cross-section. Users 
suggested that reference points need to be added to the Vertical 
Cross-Section to aid in determining the location of icing. 
Additionally, hash marks need to be added to correlate the 
Horizontal and Vertical Cross-Sections. When users zoomed in on 
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either cross-section it was very difficult to determine the 
location from one cross-section to the other. 

5.1.2.2.5 Loop Controls . 

The Loop Controls received a mode rating of 2. Users would like 
more control over the looping speed and would also like more 
choices of looping speeds. 

5.1.2.2.6 Flight Level Selectors . 

The Flight Level Selectors received a mode rating of 2. Users 
indicated they would like to be able to manually type in flight 
levels as well as using the Flight Level Selector. 

5.1.2.2.7 Data Options . 

The Data Options concept received a mode rating of 2. Comments 
indicated that users need more information for each PIREP (i.e., 
type, time, and altitude) and additional data overlays such as 
relative hiamidity. 

5.1.2.2.8 Satellite Image Selection . 

The Satellite Image Selection received a mode rating of 2. 
Overall, users indicated they need to be able to loop the 
satellite imagery in order to improve utility. 

5.1.2.2.9 Skew-T Plot . 

The Skew-T Plot was assigned a mode rating of 2. Comments 
indicated that the Skew-T display was too small and that users 
would like to see soundings from additional models as well as 
observed soundings. 

5.1.2.2.10 Volume Viewer . 

The Volume Viewer received a bimodal rating of 2 and 3. While 
the ratings were split between acceptable and borderline, the 
distribution was skewed toward the unacceptable end of the scale. 
Many users indicated the Volume Viewer had little to no utility 
and should be removed from the display. Users consistently 
indicated that the manual rotation of the display was very 
difficult and cumbersome. 

5.1.2.2.11 Viewing Angles . 

Users assigned the Viewing Angles a mode rating of 2. Most 
comments indicated that the selection of the Viewing Angles was 


11 




fairly simple; however, given the Volume Viewer had little to no 
utility, the users noted the Viewing Angles provided little 
benefit as well. 

5.1.2.2.12 Define Region . 

Users assigned the Define Region concept a mode rating of 2. No 
improvements were suggested. 

5.1.2.2.13 Selecting Vertical Levels . 

The Selection of Vertical Levels received a mode rating of 2. 

The inclusion of a reminder or better indication of the levels a 
forecaster was working with was suggested. 

5.1.2.2.14 Delete Ice . 

The Delete Ice concept received a mode rating of 2. Overall, 
user comments indicated the concept was easy to use. 

5.1.2.2.15 Add Ice . 

The Add Ice concept was assigned a mode rating of 2. Similar to 
the Delete Ice concept, user comments indicated the concept was 
useful and easy to use. It was suggested to incorporate a 
reminder of the levels where icing was being added. 

5.1.2.2.16 Vertical Interpolation . 

Users assigned a mode rating of 3 to the Vertical Interpolation 
concept. Users indicated the Vertical Interpolation was 
difficult to use. 

5.1.2.2.17 Algorithm Parameters . 

The Algorithm Parameters received a mode rating of 2. Users 
indicated this would be more useful if changes made to the 
algorithm carried through all of the forecast times for the 
particular model run. 

5.1.2.2.18 Enable Flow. 

■ 0 

Users assigned a mode rating of 2 to the Enable Flow concept; 
however, some users indicated the concept was complicated. No 
suggestions were provided for improvement. 
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5.1.2.2.19 Default Values. 


The Default Values received a mode rating of 1. No comments or 
suggestions were provided. 

5.1.2.2.20 Zoom . 

The Zoom feature received a mode utility rating of 1, but 
received a mode ease of use rating of 3. Several users indicated 
that the Zoom feature was difficult to use and quite cumbersome. 
Specifically, the two-hand activation of the Zoom feature seemed 
excessive. Users suggested a method similar to current AWC 
systems which entails positioning the cursor and then simply 
depressing the X or Z key. When zooming in the Horizontal Cross- 
Section, users commented that the Vertical Cross-Section should 
zoom in at the same ratio. 

5.1.2.2.21 Pan . 

Ratings for the Pan feature were equally divided between 2 and 3 
indicating that opinions were split between an acceptable and 
borderline rating. Users desired a feature that was less 
complicated than the current method and which would include true 
roam capabilities similar to that on the (Visual Infrared Spin 
Scan Radiometer [VISSR] Atmospheric Sounder [VAS]) Data 
Utilization Center (VDUC). 

5.1.2.2.22 Reset . 

The Reset feature received a mode utility rating of 2. Most user 
comments indicated that reset was useful and easy to use. A' 
suggestion included relocating the reset button to the menu. 

5.1.2.2.23 Undo . 

The Undo feature was assigned a mode rating of 2. It was 
suggested that an "undo all" button be implemented so a user 
could delete several actions at a time. 

5.1.2.2.24 Response Time . 

Response time acceptability received a mode rating of 3 
indicating that response time was too slow. 

5.1.2.2.25 Display . 

Display readability received a mode rating of 2; however, several 
users suggested increasing the font size. 
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5.1.2.2.26 Training. 


Although training was not formally assessed during the AIV Editor 
Evaluation, several comments regarding training were noted in the 
gliestionnaires. Users noted that they had little hands on 
experience with the system and that there was not enough written 
documentation with the display. Some users indicated that it was 
unfair to assess a system on which they had not been adequately 
trained. The lack of familiarity with the system may have skewed 
the results of the higher level functions (i.e., vertical 
interpolation, algorithm parameters). 

5.2 SCENARIO RESULTS . 

This section will document the results of the scenario sessions. 
During these sessions, evaluators noted how the editor was being 
used in order to edit an icing grid. User comments regarding the 
editor were also noted. Several forecasters indicated that while 
they had been trained to use the editor,- they were unable to 
spend extra time working on it. For this reason, they expressed 
a reluctance to provide feedback. This may have impacted some 
portions of the scenario sessions; specifically, comments related 
to higher level functions (i.e., vertical interpolation and 
algorithm parameters). 

5.2.1 Job Task Information . 

While a formal job task analysis was not completed during the 
evaluation, evaluators noted procedural steps completed during 
the scenario session. These steps may begin to outline how the 
AIV Editor would be used in an operational setting to edit icing 
grids. Procedures noted here were steps most often completed. 

It should be noted, however, that these procedures varied for 
each forecaster and reflect a generalized approach to editing an 
icing grid. Each forecaster will approach grid editing somewhat 
differently than the stated procedures. 

The following process was noted: 

a. Zoom and pan on the editor to the area of interest. 

b. Review the current analysis of icing grids presented on 
the editor in order to identify the geographical extent of icing. 

c. Display PIREPs of icing on AWIPS developed for National 
Centers. AWIPS developed for National Centers displays more 
PIREP information. The PIREPs are overlaid on a satellite image 
to see the correlation between clouds and the icing PIREPs. The 
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editor analysis is compared to the PIREPs to assess model 
performance. 

d. Review the forecasted icing grids presented on the 
editor, noting how overall features change from the analysis. 

The forecasted grids are compared to the AWIPS developed for 
National Centers displayed PIREPs. 

e. Using the scan feature of the horizontal plan view, 
examine the Vertical Cross-Section to identify the forecasted 
bases and tops of the icing layers, also making note of low-level 
icing areas and thin layers of icing. 

f. Compare the bases determined from the scanning of the 
cross-section to the forecasted freezing levels given by the 
models displayed on AWIPS developed for National Centers. Icing 
areas should not extend into areas with temperatures greater than 
freezing. Forecasters will use the freezing level from the model 
they have the highest confidence in. The choice of model varies 
from day to day. 

g. Utilize satellite animation on AWIPS developed for 
National Centers in order to determine cloud trends, such as 
growth, decay, convection, and movement. 

h. Utilize the Skew-T feature of the editor to determine 
the vertical thickness of the moist layers and where drying 
occurs. Moist layers are used to confirm forecasted areas of 
icing while drying is used to determine the height of the icing 
layer and cloud tops. Several Skew-Ts are displayed for both ice 
and ice-free regions. 

i. Display icing guidance from other model forecasts,, such 
as ETA output and the Neural Network icing product on AWIPS 
developed for National Centers. Forecasters may use the split 
screen capability of AWIPS developed for National Centers to 
perform a side-by-side comparison of different model icing 
forecasts; animate the output to display forecasted changes with 
time; or use the vertical scanning capability to identify changes 
with height. Other model forecast parameters, such as relative 
humidity, may also be used to confirm where areas of moisture 
(necessary for icing) are forecasted to occur. 

0 

j. Utilize observational data sources such as Surface 
Aviation Observations (SAOs) on AWIPS developed for National 
Centers and radar information on a dedicated monitor to identify 
areas of precipitation which, infer extensive cloud coverage. The 
location of these areas are compared with the forecasted icing 
grids. 
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k. Perform manual editing of grids. Icing grids may be 
deleted due to the lack of clouds in an area or cloud trends that 
do not correspond to editor forecast grids; the presence of icing 
grid outliers not connected to larger areas of icing; thin layers 
of icing; or icing areas that are not supported by PIREP 
information. Icing grids may be added based upon additional 
model guidance using a forecaster preferred model (e.g., ETA, 

RUC, or NGM); PIREPs; satellite images; or areas may be joined to 
form uniform areas rather than many discrete areas. 

l. After editing, the forecaster will confirm the edited 
icing grids by comparing PIREP information from AWIPS developed 
for National Centers and using the Vertical Cross-Section of the 
editor to check the vertical and horizontal extent of the 
forecasted icing grids. 

5.2.2 Comments/Enhancements for Current Editor Concepts . 

The users provided many comments during the scenario sessions. 
Comments and suggested enhancements most frequently made by users 
are discussed below. 

5.2.2.1 Training . 

While training was not formally assessed during the scenarios, 
user comments regarding training and editor familiarity were 
noted. Several users were hesitant at the start of the scenarios 
as they were unsure how to use many of the editor features (e.g., 
.algorithm parameters, vertical interpolation). Most users stated 
that the training session was not long enough. Additionally, 
many users stated that they did not have the time to utilize the 
system following the training session. The lack of familiarity 
with the system may have skewed the results of the higher level 
functions (i.e., vertical interpolation, algorithm parameters). 

5.2.2.2 PIREPs . 

Users require additional PIREP information and features. PIREP 
information should contain icing type (depicted graphically), 
flight levels, and the time of the PIREP. Additionally, users 
want the ability to overlay PIREPs in the Vertical Cross-Section. 
They would also like to have control over the amount of PIREP 
information displayed via a density control button and 
progressive disclosure as they zoom into an area. Users also 
want to be able to display PIREPs independent of grid time. 

5.2.2.3 Processing Speed . 

Although assessing system speed or performance was not an 
objective of the evaluation, users noted system processing speed 
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needs to be increased. Given the time criticality of forecaster 
tasks, the viewing and editing should be user-paced and not 
system-paced. Evaluators noted several instances where the 
forecaster could not continue with tasks because the system was 
still processing. 

5.2.2.4 Animation . 

It was noted that animation or looping of satellite and icing 
grid is essential. Forecasters use animation in order to extract 
trend information. 

5.2.2.5 Skew-T Plot . 

While the model derived Skew-T plot was found to be very useful, 
it is too small. Users had a difficult time reading the display. 
Users did not resize the Skew-T window because it would obscure 
the icing image. Users require the ability to see both the Skew- 
T and the icing grids in order to use the sounding information 
most effectively. Users also suggested the Skew-T plot be 
labeled to better indicate the geographical location of the 
model-generated sounding for the particular Skew-T and noted that 
actual sounding observations would be very useful. 

5.2.2.6 Zoom and Pan . 

The Zoom and Pan features were cumbersome and difficult to use. 
These features are used consistently and need to be very simple 
operations. They should require only one key stroke or mouse 
button depression. Additionally, users requested the addition of 
preset zoom ratios. 

5.2.2.7 Vertical Cross-Section Labels . 

Users found it difficult to determine the exact location of icing 
areas in the Vertical Cross-Section. The addition of labels, 
especially US-Canada border information was suggested by several 
users. 

5.2.2.8 Cross-Sections . 

Users found the Horizontal and Vertical Cross-Sections to be very 
helpful in identifying bases and tops, low-level icing areas, and 
thin layers of icing. The cross-sections were also useful in 
confirming changes to the grid. While the cross-sections were 
useful, users did suggest improvements. Specifically, the 
Horizontal and Vertical Cross-Section displays need to be linked 
for zooming and panning operations to ensure users are looking at 
the same area in both windows. There were several instances when 
users zoomed into a specific area on the Horizontal Cross-Section 
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and were then unable to match information to the Vertical Cross- 
Section due to the ratio mismatch. 

5.2.2.9 Display Size . 

Several users indicated that the size of the icing grid needed to 
be larger. This could be accomplished through a larger monitor 
or by changing the menus to pop-up or pull-down menus to increase 
the space already available on the screen. 

5.2.2.10 Non-Icing Grids . 

It was noted by several users that the blue grids (indicating no 
ice) degraded both the satellite imagery and the PIREP icons. 
Users indicated that they only wanted to see the red grids 
indicating the presence of ice. 

5.2.2.11 Flight Level Selector . 

At times, users found the flight level buttons to be tedious. 
Users suggested providing the option of entering flight levels 
via the keyboard. 

5.2.2.12 System Busy Indicator . 

The System Busy Indicator needs to be more apparent to the user. 
Several users did not see the System Busy Indicator and continued 
to make entries while the system was still processing. The user 
then had to continue to wait while the system processed all of 
the redundant entries. The System Busy Indicator needs to be 
prominent in the middle of the screen and lock out the ability to 
make other entries. 

5.2.2.13 Flight Level Prompt for Editing . 

While editing icing grids, several users forgot to check which 
flight levels they were editing. In several cases, users edited 
levels that were not intended to be edited. It was suggested 
that the editor prompt the user to check the flight levels before 
each edit. This will ensure that users edit only the appropriate 
flight levels. 

5.2.2.14 Font . 

The font needs to be larger to make it more readable. Many users 
had difficulty reading the text due to the font size. 
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5.2.2.15 Volume Viewer. 


Almost all of the users stated that the Volume Viewer had little 
to no utility, and they could not visualize using it in an 
operational setting. 

5.2.2.16 Looping Controls . 

While the Looping Controls were not widely used, forecasters 
stated that this feature would be useful in an operational 
setting. 

5.2.3 Enhancements for Future Iterations . 

In addition to requested modifications to current editor 
concepts, users also requested additional features. Forecasters 
provided information regarding how they would like to define 
AIRMET outlines (i.e., "From Lines") on the editor as well as 
view data over the AIRMET 6-hour forecast period. The following 
sections address these issues. 

5.2.3.1 General Enhancements . 

Forecasters requested the following enhancements: 

a. Users requested the ability to,overlay current icing 
AIRMETs and SIGMETs on the icing grids. Additionally,, users want 
to be able to click on a specific AIRMET to retrieve the • 
associated text. 

b. Users requested the ability to overlay additional 
weather information such as temperature, relative humidity, and 
freezing level. 

c. Station identifiers and coastal water boundaries need 
to be added as overlays to the editor. 

d. Many forecasters rely on models other than the RUC; 
therefore, the ability to review other weather models (e.g., NGM, 
ETA) needs to be added to the display. 

e. Users indicated that they want to be able to view data 
in a multi-panel format similar to AWIPS developed for National 
Centers. This gives the forecaster the ability to compare 
various models and forecast times. 

f. Most AWC products are valid for 6 to 8 hours; 
therefore, forecasters require the ability to loop information in 
time. A temporal looping feature should be incorporated. 
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g. Several forecasters indicated that the addition of 
icing intensity information is crucial. The current ice/no ice 
grid does not provide sufficient information. 

5.2.3.2 AIRMET From Line Creation . 

During the scenario sessions, forecasters were asked to describe 
how they would like to use the editor to create an AIRMET. Most 
forecasters indicated that they would like to be able to use a 
point and click method on the display to draw the From Line and 
then initiate a procedure that would automatically transfer the 
From Line information to the word processing program that is used 
for generating AIRMET text. Additionally, forecasters wanted the 
editor to create the state list for the AIRMET text as well. 

The automated point and click process is similar to the 
procedures utilized by the AWC Convective SIGMET forecaster. In 
the Convective SIGMET area, the VDUC allows the forecaster to 
draw the From Line around an area of interest using the point and 
click method.• Once the forecaster is satisfied with the line, a 
command is initiated to transfer the From Line information from 
VDUC to the word processing program. 

While the majority of forecasters wanted to create an AIRMET 
using the aforementioned process, some forecasters wanted the 
editor to draw an entirely automated "first guess" From Line and 
then allow the forecaster to modify the From Line by 
"rubberbanding" (i.e., reshaping or cropping). Given individual 
differences among forecasters, providing a choice of procedures 
may be beneficial. 

5.2.3.3 6-Hour Forecasting . 

Forecasters were asked how they would like to view data over the. 
6-hour valid AIRMET time period. Most forecasters indicated that 
they would like a temporal loop (i.e., animation). Some 
forecasters suggested adding the ability to create a time 
composite after editing grids over the individual forecast times. 

5.3 HUMAN FACTORS GUIDELINES AND STANDARDS COMPARISONS . 

5.3.1 Guidelines for Designing User Interface Software . 

Table 4 presents the results of the comparison of the AIV Editor 
interface characteristics to a subset of guidelines from 
Guidelines for Designing User Interface Software. 
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TABLE 4. GUIDELINES FOR DESIGNING USER INTERFACE SOFTWARE 




COMMENTS 

1. Use graphics for 
spatial and temporal 
information 

Yes 

e.g.. Satellite, grids, PIREPs, 
cross-section 

2. Use graphics for quick 
scanning and data 
comparisons 

Yes 


3. Use graphics when 
users must monitor 
changing data 

Yes 


4. Use standard format 
and labeling for all 
graphic products 

No 

Text on data options buttons 
should be standardized i.e., 

'get saved' should be called 
'retrieve', 'Print!' should have 
the exclamation point removed. 
'Hide' buttons should change to 
'show' when they are toggled on 

5. Tailor graphics to 
user needs and provide 
only necessary data 

No 

See discussions in sections 5.1 
and 5.2 

6. Highlight features 
that show critical data 

Yes 

e.g., altitude 

7. Format annotations 
consistently 

No 

PIREPs need text information for 
greater detail 

8. Display all labels in 
normal reading orientation 

Yes 


9. Establish standard 
meanings for symbols 

Yes 

Name of 'glyphs' should be 
changed to 'icons' in the User's 
Guide 

10. Design icons to 
resemble objects or 
processes they represent 

Yes 


11. Use animation for 
dynamic information 

No 

Need to loop satellite and have 
temporal loop for different 
forecast times 


Design guidelines stated as not met by the.AIV Editor are as 
follows: 


a. Guideline 4: Button labels are not standardized. For 
instance, the 'get saved' button- should be labeled 'retrieve' to 
conform to standard labeling. The 'Print!' button should not 
contain punctuation in the label. Also, since the 'Hide' buttons 
act as a toggle, the label should change between 'hide' and 
'show' to reflect the appropriate state of the button. 

b. Guideline 5: Some graphics are not tailored to user 
needs. For instance, PIREPs need additional information (i.e., 
type, time, and flight level). Additional issues associated with, 
the graphical information are discussed in section 5.2. 
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c. Guideline 7: PIREPs require text annotation to provide 
greater detail. See section 5.2.2.a. 

d. Guideline 11: Satellite imagery and icing grids need 
to be animated in order to provide forecasters with trend 
information. See section 5.2.2.C. 

5.3.2 Military Standard 1472D Guidelines . 

Table 5 presents the results of the comparison of the AIV Editor 
interface characteristics to a subset of guidelines from Military 
Standard 1472D, Human Engineering Guidelines. 

TABLE 5. MILITARY STANDARD 1472D GUIDELINES 


GUIDBLINB 

iraiilii 

COmENTS 

1. Alert/warning display 
shall provide increased 
probability of detecting 
condition 

Yes 

i.e., icing grid on/off 

2. Sufficient contrast 
shall be provided between 
display information and 
display background 

No 

Blue non-icing grids degrade 
satellite imagery 

3. Information shall be 
sufficient to allow the 
user to perform intended 
mission 

No 

See discussion in sections 5.1 
and 5.2 

4. Redundancy in the 
display shall be avoided 
unless required to achieve 
reliability 

Yes 


5. Font style should 
allow discrimination of 
similar characters such as 

1 from 1 

No 

Font is too small 


Design guidelines stated as not met by the AIV Editor are as 
follows: 


a. Guideline 2: Blue grids degrade the contrast of the 
satellite imagery. The blue grids should be removed to improve 
the clarity of the satellite imagery and PIREP icons. 

b. Guideline 3: In its current state, the editor does not 
provide forecasters with sufficient information to create an 
icing AIRMET. These issues are discussed in section 5.2. 

c. Guideline 5: The font on the display is too small and 
is difficult to read. The font should be larger. 
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5.3.3 Air Force Operational Test and Evaluation Center 
Guidelines . 

f 

Table 6 presents the results of the comparison of the AIV Editor 
interface characteristics to a subset of guidelines from Air 
Force Operational Test and Evaluation Center (AFOTEC) Software 
Usability Evaluation Guide. 

TABLE 6. AFOTEC SYSTEM USABILITY STATEMENTS 
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TABLE 6. AFOTEC SYSTEM USABILITY STATEMENTS (Continued) 


15. Menu options are easy 
to locate within menu 
hierarchy 

Yes 

16. Inactive menu 
selections are de- 
emphasized on the display 

No 

17. Display update is fast 
enough to keep up with 
user inputs 

No 

18. Display minimizes 
requirement for 
interpretation or 
memorization 

No 

19. Display formats are 
consistent across system 

Yes 

20. Wording is consistent 
across all displays 

Yes 

21. Text displays are easy 
to read 

No 

22. Display coding methods 
are consistent across 
displays 

Yes 

23. Windows are easy to 
open and close 

Yes 

24. Display symbols 
conform to accepted 
operational conventions 

Yes 

25. The use of color 
enhances display 
readability 

Yes 

26. System messages are 
informative and concise 

No 

27. User's manual provides 
complete description of 
procedures 

Yes 

28. User's manual is easy 
to use 

Yes 

30. Routine operations can 
be performed without 
user's manual 

Yes 

31. Operator workload is 
appropriate (neither too 
high or too low) 

N/A 



A user can't initiate a loop 
until loop sequence is loaded; 
therefore, loop initiation button 
should be grayed out until loop 
is loaded _ 


System response time is too slow 


Need to be able to see more than 
one forecast time on display. 
Otherwise, have to remember 
previous model data while new 
time data is loading _ 



Font is too small. 



However, blue grids degrade 
satellite image 


Error messages do not indicate 
potential solution _ 



Evaluation was not long enough to 
overcome the learning curve; 
therefore, workload was not 
assessed 
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Design guidelines stated as not met by the AIV Editor are as 
follows: 


a. Guideline 5: The system busy indicator does not 
provide the user with adequate system status. The busy indicator 
n'eeds to be more apparent to the user. 

b. Guideline 7: Current error messages do not provide the 
user with a good indication of the problem. Error messages 
should provide the user with a succinct definition of the problem 
and a potential solution. 

c. Guideline 9: The Define Regions button seems out of 
place in the edit menu. Since this concept is related to the 
algorithm parameters, it should be located in the algorithm 
parameters menu. 

d. Guideline 11: The loop controls menu and buttons are 
not ordered in the sequence of loop initiation. In essence, the 
user should select the flight levels, load the loop, and then 
initiate the loop from left to right. 

e. Guideline 13: See guideline 4 from table 4. 

f. Guideline 16: In the loop controls, a user cannot 
initiate a loop until a loop sequence is loaded; therefore, the 
loop initiation button should be grayed out until a loop is 
loaded. 

g. Guideline 17: System response time needs to decrease. 

h. Guideline 18: Forecasters need to be able to see data 
from more than one forecast time. Currently, the forecaster has 
to load the new forecast time which can take up to 10 seconds. 
Once the new forecast time is loaded, the user has forgotten the 
data from the previous forecast time. 

i. Guideline 21: The current font is too small and 
difficult to read at times. The font should be larger. 

j. Guideline 26: Error messages do not indicate potential 
solutions. 

6. CONCLUSIONS . 

6.1 GENERAL CONCLUSIONS . 

The Aviation Impact Variable (AIV) Editor presented forecasters 
with some useful concepts for viewing and editing gridded data. 
While recognizing the editor is a prototype system, several 
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improveraents are necessary in order for the viewing and editing 
concepts to be considered for operational use. The following 
paragraphs outline iitiprovements for future development. 

6.2 QUESTIONNAIRE . 

Most concepts received an acceptable utility rating; however/ 
almost all require some improvement. Both the Volume Viewer and 
Vertical Interpolation concepts received borderline ratings 
indicating that these would require significant improvement to be 
considered useful. 

All concepts except the Zoom and Pan features received an 
acceptable ease of use rating; however/ many concepts require 
some improvement. The Zoom and Pan features were rated as 
borderline indicating they require significant improvement. The 
Zoom and Pan features were cumbersome and very difficult to use. 

In general/ user comments from the questionnaire were as follows: 

a. Increase size of data window/ 

b. Increase font size, 

c. Provide additional Pilot Report (PIREP) information/ 

d. Provide temporal looping capabilities/ 

e. Provide additional weather and map overlays, and 

f. Incorporate icing intensities. 

6.3 SCENARIO . 

Results from the scenario sessions enabled evaluators to outline 
how forecasters used the editor and obtain additional feedback 
regarding the editor concepts. The procedures outlined 
represented a generalized approach to editing an icing grid. 

User suggestions for improvements of current editor concepts were 
as follows: 

a. Provide additional PIREP information; 

b. Increase system processing speed; 

c. Provide temporal looping capabilities; 

d. Increase Skew-T size; 

e. Simplify Zoom and Pan features; 

f. Add labels to cross-sections; 
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g- 

Link the Vertical and 

zooming 

and panning; 

h. 

Increase size of data 

' i. 

Remove blue non-icing 

j . 

Add manual text input 

k. 

Implement system busy 

to user 

f 

1. 

Implement check to en 

flight 

levels; 

m. 

Increase font size; a: 


Horizontal Cross-Sections for 

display window; 
grids; 

for flight levels; 

indicator that is more apparent 

lure users are editing appropriate 


n. Remove Volume Viewer. 

User suggestions for future enhancements or additions to the 
editor were as follows: 

a. Provide overlay of current Airmen Meteorological 
Statements (AIRMETs) and Significant Meteorological Statement 
(SIGMETs); 


b. Provide additional weather and map overlays; 

c. Provide access to other forecast models; 

d. Implement multi-panel format to view data; 

e. Incorporate icing intensity information; and 

f. Implement point and click method for AIRMET From Line . 
creation with automatic data transfer from the editor to word 
processing program. 

6.4 HUMAN FACTORS STATEMENTS AND GUIDELINES . 

The AIV Editor display and interface follow many human factors 
standards and guidelines; however, both the display and interface 
require improvement in several areas as noted in paragraph 5.3. 
The interface should reflect job tasks and follow good design 
principles. A usable interface can improve performance and 
decrease forecaster workload and error during critical weather 
conditions. 
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6.5 TRAINING. 


Many users indicated that training was inadequate. The length of 
training was insufficient to adequately train forecasters. 
Additionally, most forecasters did not have time to work on the 
editor after training. Consequently, many forecasters were 
uncomfortable with several editor functions; specifically, the 
Vertical Interpolation and Algorithm Parameter functions. This 
may have impacted the forecasters' opinions of these higher eve 
functions. 

7. RECOMMENDATIONS. 


a. User job tasks should be defined prior to system 
development. The development agencies must have a thorough 
understanding of the user job tasks and information requirements 
in order to. develop systems that are oriented to the user rather 
than expecting the user to adapt to a system. 

] 3 , All users should be trained to an acceptable level of 
performance prior to the start of any evaluation. Additionally, 
adequate time between training and the evaluation for user 
familiarization, thus, reducing biases resulting from users 
operating within the learning curve. 

c. Future iterations of the editor should incorporate 
features from current AWC operational platfo^s and specifically 
include the following currently available editing and viewing 
concepts of the AIV Editor: 

1. Horizontal and Vertical Cross-Sections, 

2. Manual editing capability, 

3. Overlay capability, and 

4. Spatial animation. 

d. Future iterations of the editor should have the 
following enhancements that are not a part of the current AIV 
Editor: 

1. Temporal animation of satellite and model forecast 

grids; 

2. Additional PIREP information, including type, 
time, and altitude; 

3. Additional overlay capabilities of weather 
information such as relative humidity, temperature, freezing 
level, AIRMETs, and SIGMETs; 
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4. The editing must be portable so that grids from 
different models may be edited upon the same platform. 

e. Editor enhancements should be prioritized to fill the 
most urgent needs of AWC forecasters first. 

f. Future iterations of the editor should follow standard 
human factors guidelines for the presentation of information. 
Suggested guidelines include Human Factors Engineering Guidelines 
{MIL-STD-1472D) and Guidelines for Designing User Interface 
Software (ESD-TR-86-278). 

g. Based upon a lack of strong user identified utility, no 
further development work should be conducted on the three— 
dimensional volume viewer. 

h. Further assessment regarding the utility of the 
manipulation of algorithm parameters needs to be performed in 
order to proceed with further development. ^ 



8. ACRONYMS. 


AFOS 

AFOTEC 

AIRMET 

AIV 

AVS 

AWC 

AW I PS 

EFF 

FAA 

FSL 

NCAR 

NCEP 

NGM 

NOAA 

NWS 

PIREP 

RAP 

RUC 

SAO 

SIGMET 
VAS 
VDUC 
VIS SR 


Automation of Field Operations and Services 

Air Force Operational Test and Evaluation Center 

Airmen Meteorological Statement 

Aviation Impact Variable 

Application Visualization System 

Aviation Weather Center 

Advanced Weather Integrated Processing System 

Experimental Forecast Facility 

Federal Aviation Administration 

Forecast Systems Laboratory 

National Center for Atmospheric Research 

National Centers for Environmental Prediction 

Nested Grid Model 

National Oceanic and Atmospheric Administration 
National Weather Service 
Pilot Report 

Research Applications Program 

Rapid Update Cycle 

Surface Aviation Observation 

Significant Meteorological Statement 

VISSR Atmospheric Sounder 

VAS Data Utilization Center 

Visual Infrared Spin Scan Radiometer 
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APPENDIX A 


QUESTIONNAIRE DATA 




VIEWING FUNCTIONS: 


Data File Selector 

1. Rate the utility of the data file selector. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


2. Rate the ease of use of the data file selector. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


3. List suggestions for improving the data file selector. 


2 

12 

0 

0 

0 

0 


1 

11 

1 

0 

0 

0 


a. Too small and too fine print (font) 

b. Data was already loaded, difficult to really judge 

c. Would like verifying times added. Selector having more 
lines displayed would be nice 

d. Needs to indicate valid time 

e. Need a feature where you could view data over a selected 
time interval i.e., 12-15Z; ability to loop as well 

f. Should be able to load a loop with SAT off 


A-1 





Icing Category Selector (Display Options) 

1. Rate the utility of the Icing Category Selector. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


2. Rate the ease of use of the Icing Category Selector. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


3. List suggestions for improving the Icing Category Selector 

a. Icing should be in intensities 

b. Of more value would be to incorporate Don McCann's 
neuronet program to differentiate between rime, mixed, or 
clear icing potential 

c. Display is OK as is 

d. Some relation to icing types would be useful 

e. Intensity levels would be good 


Horizontal Cross-Section 

1. Rate the utility of the Horizontal Cross-Section. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 







2 . 


Rate the ease of use of the Horizontal Cross-Section. 

1 
9 
4 
0 
0 

_0 

3. Rate the readability of the Horizontal Cross-Section. 

0 1 - Completely Acceptable 

8 2 - Acceptable 

4 3 - Borderline 

2 4 - Unacceptable 

0 5 - Completely Unacceptable 

0 NA 


4. Rate the acceptability of the update speed of the Horizontal 
Cross-Section. 


1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


1 

9 

3 

0 

0 


1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


5. List suggestions for improving the Horizontal Cross-Section. 


a. Model data should be in computer's memory 

b. Too slow — Poor colors. Need quicker response and more 
color selections 

c. Needs a delicate touch on the mouse or you "define a 

region" instead . ' 

d. It's easy to get ahead of the computer 

e. The only labels to identify your position are at either 
end with nothing in between 
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f. Would like to see bigger image, also grid overlay tends 
to hide some of the clouds on satellite image 

g. Delete blue grids. Provide for operator to change 
colors; i.e., color blind operators 

h. Need a feature where zoomed image corresponds to cross- 
section. Also would be helpful to see a cross-section from 
just the area of interest 

i. Intensities would be good 


Vertical Cross-Section 



Rate the utility of the Vertical Cross-Section. 

1 1 - Completely Acceptable 

12 2 - Acceptable 

1 3 - Borderline 

0 4 - Unacceptable 

0 5 - Completely Unacceptable 

0 NA 


2. Rate the ease of use of the Vertical Cross-Section. 


0_ 1 - Completely Acceptable 

8_ 2 - Acceptable 

5_ 3 - Borderline 

1_ 4 - Unacceptable 

0 5 - Completely Unacceptable 

0 NA 


3. Rate the readability of the Vertical Cross-Section. 

0 1 - Completely Acceptable 

10 2 - Acceptable 

'3 3 - Borderline 

1 4 - Unacceptable 

0 5 - Completely Unacceptable 

0 NA 
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4. 


Rate the acceptability of the update speed on the Vertical 
Cross-Section. 


1 

13 

0 

0 

0 

0 


1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


5. List suggestions for improving the Vertical Cross-Section. 


a. Show intensities 

b. Feature to step through flight levels would be helpful. 
Again need feature to match vertical and horizontal cross- 
sections if zooming 

c. More difficult to identify location. Needs some 
reference points 

d. Tie to zoom of horizontal cross-section. Need altitude 
labels retained when zoomed. No need of zooming in 
vertical, only horizontal 

e. Add PIREPs to cross-section. Add some kind of hash mark 
to orient viewer between horizontal and vertical cross- 
sections. Link horizontal and vertical cross-section so 
that if you blow up or enlarge horizontal portion the 
vertical cross-section covers the same area (Only 1 to 1 at 
default setting) 

f. Need to be able to set bounds to limit zoom area to 
better integrate area of intensities 

g. Computer needs to be faster 

h. Might consider adding diagonal cross-section capability 

i. You need to be able to go directly to where you want a 
vertical cross-section rather than scan to it. Match with 
horizontal view especially when zooming 

j. North/South not as useful as East/West 
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Loop Controls 


1. Rate the utility of the Loop Controls. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


2. Rate the ease of use of the Loop Controls. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 

3. Rate the acceptability of the animation speed on the Loop 
Controls. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


4. List suggestions for improving the Loop Controls. 


a. Need choice of speeds 

b. Might add a forward/reverse option 

c. Crash 

d. Loop of most recent 8 images or so should always be 
loaded so that looping is just turning on and off 

e. Need windshield wiper mode 






f. Little need for vertical looping with availability of 
vertical cross-section. Need a time looping capability 

g. Speed seems too fast. Would be nice to be able to 
control speed of loop 


Flight Level Selectors 


1. Rate the utility of the Flight Level Selectors. 



1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


2 . 


Rate the ease of use of the Flight Level Selectors. 


5_ 1 - Completely Acceptable 

8_ 2 - Acceptable 

1_ 3 - Borderline 

0_ 4 - Unacceptable 

0_ 5 - Completely Unacceptable 

0 NA 



3. List suggestions for improving the Flight Level Selectors. 

a. Why not just type in flight levels instead of long list 
of flight levels? 

b. Though well displayed, sometimes forgot levels selected 
when doing .an operation 

c. Nice 
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Data Options 

1. Rate the utility of the Data Options. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


2. Rate the ease of use of the Data Options. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 

3. List suggestions for improving the Data. Options. 

a. Might consider more options such as RH values (mean or 
at different levels) 

b. PIREPs need type and altitude data 

c. OK as is, need more data types to be available for 
display 

d. Need to be able to remove blue grid while retaining red 
grid 

e. PIREP data somewhat difficult to read 

f. Need more information on PIREPs. Could add more fields 
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Satellite Image Selection Options 

1. Rate the utility of the Satellite Image Selectors 

• 2 1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


10 


Rate the ease of use of the Satellite Image Selectors 

2 1 - Completely Acceptable 

9 2 - Acceptable 

2 3 - Borderline 

0 4 - Unacceptable 

0 5 - Completely Unacceptable 


NA 


3. List suggestions for improving the Satellite Image 
Selectors. 

a. Need looping or more looping capability 

b. Would be nice to have satellite looping capability 

c. Was not functioning during my session due to 
communication problems 

d. Simple enough 

e. Would like to be able to loop satellite images 


Skew-T Sounding Plot Selector 

1. Rate the utility of the Skew-T Sounding Plot Selector. 


3 

8 

3 

0 

0 


1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 
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Rate the ease of use of the Skew-T Sounding Plot Selector. 


2 

S 

3 

0 

1 

0 


1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


List suggestions for improving Skew-T Sounding Plot 
Selector. 


a. Need more models and a way of changing soundings 

b. Remove 3-D contour option 

c. Too small 

d. Would be used more often with a larger display area 

e. Larger font size for easier readability. Zoom feature 
is nice but would be helpful to have zoomed skew-t adjacent 
to Satellite picture and cross-sections for comparison 

f. Could add stability indices 
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Three Dimensional Volume Viewer 


1. Rate the utility of the Three Dimensional Voliime Viewer, 


1 _ 1 - Completely Acceptable 

4_ 2 - Acceptable 

4_ 3 - Borderline 

3_ 4 - Unacceptable 

2 5 - Completely Unacceptable 

0 NA 


2. Rate the ease of use of the Three Dimensional Volimie Viewer. 



1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


3. Rate the readability of the Three Dimensional Volume Viewer. 


1 _ 

2 

2 

0 


1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


4. List suggestions for improving Three Dimensional Volume 
Viewer. 


a. Need more intensities 

b. Helpful feature but would take a while to get used to. 

Is it possible to zoom in? Could PIREPs or even temperature 
be plotted some way on a 3-D image? 

c. Difficult to use without lots of practice. Don't see 
how it can be applied to our type of product 

d. No operational need or use 

e. Personally, I don't see the utility of this for the 
forecaster. Nice display but better for a conference 
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presentation or a TV meteorologist rather than an 
operational forecaster 

f. Not much utility in use. Nice to see type thing 

g. Cute, but the value of manual rotation is not clear 

h. Neat idea but hard to handle with mouse 

i. Questionable value at best. Vertical cross-section 
better tool and performs basically same function. Difficult 
to manipulate. Would prefer automatic rotating controls 

j. Forget it! 

k. Suggest this be dropped 


Viewing Angles 

1. Rate the utility of the pre-set Viewing Angles. 

0 1 - Completely Acceptable 

8 2 - Acceptable 

4 3 - Borderline 

2 4 - Unacceptable 

0 5 - Completely Unacceptable 

0 NA 


2. Rate the ease of use of the pre-set Viewing Angles. 



1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 
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3. Rate the utility of the user-specified Viewing Angle. 


3 

8 

3 

0 

0 


1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


4. List suggestions for improving the Viewing Angles. 


a. Questionable value at best. Vertical cross-section is a 
better tool 

b. Not bad 

c. 3D limited utility on 2D screen 

d. Since I question the utility of the three dimensional 
volume viewer, I also question the utility of this product 
in an operational environment 

e. No operational need or use 

f. Hard to orient icing areas with terrain when viewed at 
low angles 


EDITING FUNCTIONS 
Define Region 


1. Rate the utility of the Define Region feature. 



1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


A-13 




2. Rate the ease of use of the Define Region feature. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA . 


3. List suggestions for improving the. Define Region feature. 


a. Too hard to figure out 


Selecting Vertical Levels 

1. Rate the utility of Selecting Vertical Levels. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


2. Rate the ease of use of Selecting Vertical Levels. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 

3. List suggestions for improving the selection of Vertical 
Levels. 

0 

a. Nice 

b. Need requester or other clue to remind forecaster which 
levels they are working with 

c. Found it somewhat cumbersome to understand and use 



2 

9 

3 

0 

0 

0 
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Delete Ice 


1 . 


Rate the utility of the Delete Ice feature. 


2 

10 

0 

1 

1 

0 


1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
' NA 



Rate the ease of use of the Delete Ice feature. 


2 

11 

0 

0 

1 

0 


1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


3. List suggestions for improving the Delete Ice feature. 


a. Easy to use 

b. Works well 

c. Fine 

d. Why go against meteorology 

e. Is imperative that this be integrated into the forecast 
phase 


Add Ice 


1. Rate the utility of the Add Ice feature. 


2 

10 

1 

0 

1 

0 


1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 
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2. Rate the ease of use of the Add Ice feature. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


3. List suggestions for improving the Add Ice feature. 

a. Is imperative that this be integrated into the forecast 
phase 

b. Need way to be sure forecaster is adding ice in areas 
where they want them 


Vertical Interpolation 

1. Rate the utility of the Vertical Interpolation. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


2. Rate the ease of use of the Vertical Interpolation. 


1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 
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3. 


List suggestions for improving the Vertical Interpolation 
feature. 

a. Didn't encounter this feature 

b. It's a little tricky to use 

c. Could not make it work 


Algorithm Parameters 

1. Rate the utility of the Algorithm Parameters. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


1 

B 

£ 

0 

0 

1 


2. Rate the ease of manipulating the slider bars on the 
Algorithm Parameters. 

1 1 - Completely Acceptable 

8 2 - Acceptable 

3 3 - Borderline 

1 4 - Unacceptable 

0 5 - Completely Unacceptable 

1 NA 


3. List suggestions for improving the manipulation of the 
Algorithm Parameters. 

a. Fine 

b. Need other ways at getting at data to manipulate model 
output so that feedback goes back to the model and other 
parameters change accordingly 

c. How about if you change an algorithm parameter at a 
certain point of a RUC run that that change continue on 
through the remainder of the grids. For example, off the 
12Z RUC, you change a temperature or RH on the 15Z grid to 
retain those changes on the 18Z, 21Z, and OOZ grids? 

d. Did not use this. Need to work with it more 
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Enable Flow 


1. Rate the utility of the Enable Flow feature. 


£ 

2 

3 

0 

0 

2 


1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


2. Rate the ease of use of the Enable Flow feature. 


£ 

_9 

2 

2 

2 

0 


1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


3. List suggestions for improving the Enable Flow feature. 


a. Did not use 

b. Complicated 


Default Values 


1. Rate the utility of the Default Values feature. 

7 1 - Completely Acceptable 

6 2 - Acceptable 

0 3 - Borderline 

0 4 - Unacceptable 

0 5 - Completely Unacceptable 

1 NA 


A-18 



2. ' Rate the ease of use of the Default Values feature. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


3. List suggestions for improving the Default Values feature. 


a. Fine 


FUNCTIONALITY: 

Zoom 

1. Rate the utility of the Zoom feature. 

8 1 - Completely Acceptable 

4 2 - Acceptable 

2 3 - Borderline 

0 4 - Unacceptable 

0 5 - Completely Unacceptable 

0 .. NA 

2. Rate the ease of use of the Zoom feature. 

0 1 - Completely Acceptable 

4 2 - Acceptable 

6 , 3 - Borderline 

2 4 - Unacceptable 

2 5 - Completely Unacceptable 

0 NA 


3. Identify Zoom features that could be improved and provide 
suggestions for improvement. 

a. Shift-middle button is clumsy 

b. Awkward having to use 2 hands — should be able to zoom 
in/out centered at a specified point 

c. Requiring two hands to zoom seems excessive 
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d. Very important feature 


e. Somewhat cumbersome to hold down shift-key and move 
cursor either up or down. How about like NTRANS set the 
point then either hit a X or Z key? 

f. Change to use 2 mouse keys instead of a mouse and a 
keyboard key. Need to tie vertical screen to horizontal 

g. Preset zoom values to zoom step by step would be 
helpful; just by clicking mouse button; left for zooming; 
right for unzooming 

h. Cumbersome (should be done with one button) 


Pan 

1. Rate the utility of the Pan feature. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


2. Rate the ease of use of the Pan feature. 


1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


3. Identify Pan features that could be improved and provide 
suggestions for improvement. 

a. Fine 

b. Needs to be true roam feature with image moving as mouse 
cursor moves 
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c. Cumbersome to use 

d. Should be an easier way 


Reset 


1. Rate the utility of the Reset feature. 



1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


2. Rate the ease of use of the Reset feature. 

3 1 - Completely Acceptable 

8 2 - Acceptable 

3 3 - Borderline 

0 4 - Unacceptable 

6 5 - Completely Unacceptable 

0 NA 


3. Identify Reset features that could be improved and provide 
suggestions for improvement. 


a. Prefer a menu button 

b. Put reset image button, on AIV workstation screen. Too 
easy to,forget F7 ’ 

c. Easy enough 

d. What are we doing here? Evaluating hitting the F7 
button 
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Undo 


1. Rate the utility of the Undo feature. 


1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


2. Rate the ease of use of the Undo feature. 

1 - Completely Acceptable ' 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 

3. Identify Undo features that could be improved and provide 
suggestions for improvement. 

■ a. A button "undo all" when in editing mode would reduce 
number of steps needed to undo a line 

b. Multiple level undo would be handy. Some editor word 
processors have scroll bar to work back through your changes 

c. Easy 


Response Time 

1. Rate the acceptability of the response time (i.e., time 
between data entry and data presentation) of the 
products/functions. 

1 - Completely Acceptable 

2 - Acceptable 

3 - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 






2. Identify any product/function in which the response time 
could be improved. 

a. Add/delete ice 

b. Computer needs to be faster 

c. Need more feedback to let forecaster know something is 
happening 

d. Remove 3-D Volume Viewer 

e. Loading of data files; model data and time projections 

f. Overall slow 


DISPLAY: 


1. Rate the overall readability of the display. 


1 

9 

2 

1 

0 

1 


1 - Completely Acceptable 

2 - Acceptable 
3. - Borderline 

4 - Unacceptable 

5 - Completely Unacceptable 
NA 


2. List any suggestions for improving the readability of the 
display. 

a. Too small and no way of reading individual reports. 
Need larger satellite picture and better data availability 

b. Larger 

c. Scalability of fonts and ability to resize and move 
things important for those with less than perfect eyesight 

d. None 

e. Larger screen. Put algorithm parameters in a pop up 
window to save screen space 
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f. More contrast to gray entries on left side of screen 
would be easier to read. Icing areas displayed in a solid 
color would be easier to see than cross hatched 


3. Is there any product information in a graphical format that 


you 

would prefer in a textual 

format? 


a. 

No 



b. 

Actual observations and PIREPs 


c. 

No 



d. 

PIREPs 



e. 

the 

It would be nice to click 
original 

on a PIREP 

and see the text in 

f. 

No 



g- 

Not that I'm aware of 


•• 

h. 

Ability to add tops/bases 

to PIREPs, 

and PIREPs of no 


icing and reports not mentioning icing 

i. PIREP information. Icing potential areas displayed on 
vertical and horizontal cross-sections siammarized in textual 
format 


4. List any colors that you had difficulty differentiating and 
provide suggestions for color changes. 

a. None 

b. State boundaries to light when data is displayed. 

Perhaps be drawn over grid 

c. None 

d. Satellite image enhancement and grid interferes with 
each other. Would likely change grid color or turn off grid 
in real-time use 

e. Satellite pictures difficult to see 
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f. Satellite picture hard to understand since colors not 
dark enough (bright enough?). Whole display is very poor 


5. Are there any other features that you would like added to 
the system? 

a. Add From line by pointing and clicking and send to a 
word processor 

b. Is it possible to overlay radar composites on Satellite 
picture or horizontal/vertical cross-sections? 

c. Looping displayed icing areas to see how areas change 
with time. Overlaying icing areas to get total threat area 
for a 6 hour valid period 

d. VOR locations, another screen to show a different ■ 
forecast time or model to compare. Need the active mode 
indicator moved to top panel and emphasized so you know 
system is responding to your mouse click 

e. Not that I can think of at this time 

f. More real time data sources such as SAO's and UA's 
freezing level information, composite radar imagery, cloud 
top information from Satellite imagery 

g. Need at least one more screen for data (PIREP, SAO, UA, 
etc...) 

h. . More than on satellite image 

I. Icing intensities (better algorithm) 


GENERAL; 

Please provide any other comments or suggestions you may have 
regarding the functions, display, and/or system. 

a. None 

b. TO be an all inclusive system; would need to predict 
type and intensity of icing as well. Would also need 
versatility of applying RAP algorithm using ETA, Meso-ETA, 
NGM, AVN models. Overall, system has good possibilities. 
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RAP algorithm did predict fairly closely a couple of areas 
of mid/high level icing potential where models missed it 

c. In general a very good system but has a few fancy 
displays of little practical forecasting use. Needs time 
looping capabilities and time/model comparison method 

d. I had little on hands experience with the AIV editor. I 
did sit down a few times after the training to become more 
acquainted with it. However, I did not participate in the 
case scenario 

e. Need to provide ways to interpolate from one forecast 
period to another and follow the process to ensure 
consistency and accuracy. This could be done by looping 
finished forecast products, etc... 

f. For a first attempt has some value but is very far from 
being better than what we already have 

g. You develop a system and send someone out to train the 
users... but not much written documentation on use 

h. To require an evaluation with less than 2 hours of 
training (hands on training was less than 15 minutes) is 
grossly unfair to the system and what is trying to be 
accomplished 
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APPENDIX B 


PROCEDURES AND USER COMMENTS FROM 
SCENARIO SESSIONS 




Forecaster: 1 


Scenario 1: 

1. Description: Evaluation time of 15Z (9:00 am local). Icing 
potential valid for 18Z for MIA area. 

2. Procedures: 

Look at current analysis. 

Use N-AWIPS to examine most recent PIREPs (within 3 hours). 
Identify severity, icing type, horizontal and vertical 
location. 

Look at valid time. 

Zoomed on southeast US and panned to center region of 
interest. 

Used cross-section to manually scan through plan view and 
examine cross-section to identify bases and tops of icing 
areas. 

Compare bases with freezing level. Contoured freezing level 
acquired off N-AWIPS for valid time using RUC (same model as 
editor). Use freezing level to determine icing bases. 

■ Identify icing areas to delete. Use satellite to determine 
where cloud systems have already moved eastward out of icing 
areas. 

Ignore outliers where only one or two grids are identified, 
to have icing. Regions do not meet AIRMET size criteria. 

Compares valid time (18Z) with initial conditions (12Z) to 
determine if and how conditions are changing. 

Deletes icing from cloud free regions behind satellite- 
identified cloud system. Uses manual deletion of icing 
grids. 

Use skew-T option to determine thickness of moist layers. 
Viewed several locations within icing area and cloud shield 
from satellite. Based upon skew-T observations, identified 
drying out of the atmosphere on western portions on cloud 
system. Manually deleted icing from western portion 
(trailing edge) of cloud system. 
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Manually extended icing area to include portions of cloud 
system that were known to be producing rain based upon 
previous understanding of current weather conditions. 
Vertical extent of additional icing area coincides with 
previously identified bases and tops. 

Scenario 2: 

1. Description; Same evaluation time. Valid time also of 18Z/ 
but for SFO area. 

2. Procedures: 

Zoom and pan to center area of interest. 

Examine cross-section manually to determine bases and tops. 
Compare with freezing level from N-AWIPS. 

Examine several skew-Ts within area of clouds, but with no 
icing potential predicted. Examining depth of moisture. 

Compares with PIREPS on N-AWIPS. 

Uses satellite animation on N-AWIPS to determine short term 
trends in cloud motion. Decides to remain ice-free 
forecasted region as is since satellite animation confirms 
moving out of area. 

Leaves, automated icing grids as is and would represent with 
one AIRMET. Original thought was to have two AIRMETs, but 
instead decided upon one area of icing based upon latest 
PIREP information and satellite animation trend which showed 
more clouds entering region. 

Further examination of different icing classes (specifically 
stable and freezing drizzle—only classes of interest) on 
N-AWIPS, decides to have two separate areas of icing. One 
region will have rime, other region will have mixed and 
clear. Determination was made by animating model forecasts 
on N-AWIPS and also using four panel display which allowed 
comparison of different classes. Stable regions used for 
rime icing; freezing drizzle used for mixed and clear. 
Compared regions to PIREPs to confirm or modify. 

Comments and Enhancements: 

PIREPs need to emulate N-AWIPS display, specifically need to 
include symbol, type, and flight level. 
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Outlying icing areas which are too small for AIRMET criteria 
should be removed. 

Speed needs to be increased. 

Display freezing level as a contour option. 

Need animation of satellite images in order to determine 
short term trends (such as speed of cloud systems). VDUC 
capable of clicking on beginning point and end point to 
determine speed and extrapolate future position. Preferred 
to animate last 6 hours of images. 

Would prefer satellite images next to editor to decrease 
clutter, but still include animation under icing grids. 

Animate icing grids to coincide with satellite animation. 

Can determine model trends. 

Should emulate N-AWIPS capability to animate and have 
multiple panels displayed to look at different classes of 
algorithms and different valid times. Can not directly 
compare different classes (e.g., freezing drizzle and 
stable). 

Should be able to overlay and differentiate between 
different algorithm classes. 

Pre-set views of volume viewer are beneficial. Can 
determine sloping surfaces and outliers. 

Vertical looping is beneficial for determining bases and 
tops. 

Display current advisories for comparison with model 
forecasts. 

Cross handed placement of current system is awkward. Need 
to make sure that the mouse is to the right of the keyboard. 

Cross-section is extremely useful in picking out top layers 
of icing. 

0 

Would like to overlay current AIRMETs to compare information 
and be able to overlay PIREPs for the old AIRMET times. 

Freezing level at surface, would like the topographic 
database. 

The adding and deleting ice tools are very useful. 


B-3 



Wants to be able to compare stable and freezing drizzle 
algorithms similar to 2 panel display on NTRANS. 

When looking at algorithm parameters. Would like to look at 
algorithm parameters to issue two separate AIRMETs based on 
type. 

Would have used loop controls but didn't remember how to use 
it. 

Liked pre-set'views but didn't like to grab with mouse and 
rotate around. 

To make an AIRMET: 

Need location (VOR) identifiers or markers. 

Need boundary between domestic and international areas. 

Point and click to identify AIRMET area. 

Transfer automatically to PC (as with current VDUC 
capability for Convective SIGMETs). Also include states. 

Vertical extent of AIRMET would be entered manually based 
upon layers identified in editor. 

Would also like to define states in the AIRMET as well as 
the station identifiers. 

For a 6-hour valid AIRMET: 

Edit 3, 6, and 9 hour forecasts (or whatever falls into 
valid block) and then form unison (composite). Hourly data 
probably too much data and would increase forecaster 
workload. 

Would like to be able to bring up information across set 
times and be able to loop through the information. 

Show grids at each of the time steps all overlaid. 
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Forecaster: 2 


Scenario 1: 

1. Description: SLC area; analysis time of 19Z; valid time of 
21Z. 

2. Procedures: 

In horizontal plan view, go to northernmost point of area of 
interest. Use North to South scan to review cross-section 
in order to identify vertical extent of regions. 

Examine several skew-Ts in icing regions to confirm depth of 
moisture as shown in cross-section. Especially noting at 
what levels drying occurs—important for determining tops of 
icing layers. 

Look at PIREPs. Used AFOS on N-AWIPS to review last 3 
hours. Needed flight level of PIREPs—not available on 
editor. Also, editor does not provide time identification 
of PIREP. Compare PIREPs to information on vertical extent 
from skew-Ts. 

Look at gaps in horizontal plan view of icing. Are these 
realistic based upon PIREPs? 

Low lying icing areas identified by scanning and vertical 
• cross-section are checked by reviewing SAOs in region of 
interest. SAOs show warm temperatures and downsloping winds 
which lead to drying. Deleted areas manually. Confirmed 
downsloping winds by looking at RUC 850 mb. wind field. 

Changed to 21Z forecast grid. Scanned North to South in 
order to identify bases and tops from cross-section. 

Looking for agreement with earlier analysis. 

Manually removed outliers (isolated extensions) of larger 
icing regions. 

Scenario 2: 

1. Description: CHI area, valid time of 21Z, analysis time of 
15Z. 

2. Procedures: . 

Look at RUC neural network output on N-AWIPS. 
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Use editor to scan N-S on 15Z analysis in order to become 
aware of general situation. 

Change to 21Z forecast with PIREPS of +/- 3 hours plotted. 

• Look at SAOs on AFOS that coincide with icing areas. 

Checking cloud observations for agreement with icing areas 
on editor. 

Look at PIREPS on N-AWIPS that include height information. 

Use skew-T on editor to identify moist levels. Looking in 
both icing and ice free regions. 

■ Manually deletes a small area separate from large icing 
region. PIREPs were only in large region. Skew-t show only 
shallow moist layer in deleted region. 

Look again at SAOs for region around periphery of icing 
region. Cloud cover is not substantial. Radar does not 
show any precipitation (which would indicate substantial 
cloud cover). 

Manually delete areas that feel are marginal based upon 
previous step. 

Scan N-S checking bases and tops. 

Satisfied with area based upon support from SAOs, PIREPs, 
and radar. 

Comments and Enhancements: 

Cross-section is very useful, especially when combined with 
scanning. 

Vertical looping would be useful. Did not use due to 
unfamiliarity. 

Time savings because graphical forecast is already made. 
Forecaster does not have to perform analysis. 

Volume Viewer: May use as become more comfortable with 
manipulating image. Would probably use after editing grids 
as an auxiliary tool to display effect of editing. However, 
benefits are unknown currently. 

Algorithm Parameters: Would rely upon defaults until 
realize something needs changing based upon experience with 
. model. 



Needs more information on PIREPs. 

Would like to display PIREPs on cross-section. 

Prefers separate display systems in order to compare 
products. 

Would like to see more moisture information. 

To make an AIRMET: 

Point and click to outline area. 

Electronic transfer of points to PC. Also include states 
(effectively making first two lines of AIRMET). 

For a 6-hour valid AIRMET: 

Need animation through time periods. 

Look at skew-Ts for each time during the valid period. 

Want to have capability to step through each time during the 
valid period. 
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Forecaster: 3 


Scenario 1: 

1. Description: No product. 

2. Procedures: 

Forecaster uncomfortable with editor. Not enough hands-on 
use. 

Reviews editor features and provides comments. 

Scenario 2: 

1. Description: Icing potential valid for 21Z for SLC area. 

2. Procedures: 

Look at PIREPs on AFOS (comfortable with AFOS display). 

With editor, looking at satellite image. Hiding icing grids 
because grid degrades image. 

Scan N-S and return. 

Using N-AWIPS to look at neural network icing product. 

Using composite to get idea of where icing is being 
predicted. Compare with PIREPs on AFOS. 

Look at 700 mb relative humidity on N-AWIPS to determine 
where is the forecasted moisture. Trying to determine non- 
moist areas that may be predicted by icing algorithm. 

Manually delete icing grids ahead of cloud system since feel 
cloud systems are not moving as fast as what RUC predicts. 

Use cross-section on editor to check icing tops and bases. 

Based upon forecast shift, knows icing is at high levels. 
Manually adds icing grids to increase vertical extent. 

Comments and Enhancements: ' 

Cross-section is beneficial with scanning option. 

Would like to see PIREPs in the vertical cross-section. 

Some type of identification markers on vertical cross- 
section would be helpful in order to know location. 
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Mouse is too sensitive. 

Would be beneficial to display current AIRMETs.' 

■ Grids every 3 hours are good. Run risk of data overload 
with more data unless tremendous improvements in model 
accuracy are achieved. 

Interface is intimidating due to number of options. 

More comfortable with typing flight levels of interest. 

Time is critical. Can not devote a lot of time to editing. 
Other forecast problems must be dealt with. 

Cross-section needs to be synchronized with horizontal plan 
view when zoomed. 

More screen real estate for satellite image would be 
helpful. 

Retain latest PIREPs on editor view regardless on image time, 
rather than only a window around image time. Similar to 
satellite image concept. 

Would like to see model relative humidity fields overlaid on 
editor icing grids. 

No utility for volume viewer. 

Looping controls appears as if would be helpful by the 
defining of interested levels. N-AWIPS only allows single 
level views or complete composite. Were not used due to a 
lack of familiarity. 

Wants to see display size increased. Has suggested that 
this may be accomplished by removing menus from screen. 

To make an AIRMET: 

Include VORs on editor. 

Point and click with points transferred to PC. 

For a 6-hour valid AIRMET; 

Split screen to see beginning and ending of valid period. 
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Forecaster: 4 


Scenario 1: 

1. Description: Forecaster not comfortable with editor. 

2. Procedures: Walk through features. 

Scenario 2: 

1. Description: Still uncomfortable with editor. 

2. Procedures: Walk through features. 

Comments and Enhancements: 

Plotting of PIREPs is important. 

Allow individual to define pre-set colors. Some forecasters 
are color-blind. 

Icing grid degrades satellite image. Perhaps only show 
icing grids. If no icing exist/ then don't show grid. 

No utility for volume viewer. 

Bracket critical temperature regime for icing (0°C and 
20°C) . 

Finds the stability indices very useful. 

PIREPs need additional information (i.e., Type, Severity, 
Time). 

Would like to see temporal looping. 

Would like to be able to have option for text inputs to 
select flight levels. 

Critical temperature overlays would be useful. 

To make an AIRMET: 

Point and click which would transmit points automatically to 
PC. 

For a 6-hour valid AIRMET: 

Wants to be able to loop through time models. 
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Forecaster: 5 


Scenario 1: 

1. Description: SLC area with a valid time of 21/OOZ due to 
data problems with editor. 

2. Procedures: 

Compare with N-AWIPS. Look at icing product on Eta for 
valid time. 

Use paper VOR map to sketch outline of icing area from'Eta. 
Look at RUC icing product for valid time. 

Look at RUC neural net icing product. 

Look at 500 mb vorticity product on N-AWIPS to identify 
short waves.. 

Look at relative humidity for 900, 700, and 500 mb to 
identify major areas of moisture. Compare with editor icing 
grids. 

Looked at location of editor displayed PIREPs. 

Manually delete icing grids that felt were an overforecast 
based upon information viewed on N-AWIPS and PIREPs. 

Scenario 2: 

1. Description: BOS area with a valid time of 21/09Z. No 
forecast information available due to data problems. 

2. Procedures: 

Compare editor icing grids to RUC neural network output on 
N-AWIPS. Seeing if agreement in areas is reasonable. 

Use split panel capability of N-AWIPS to compare 09 and 12Z 
icing products to determine movement and changes. 

« 

Use zoom and pan functions of editor to display area of 
interest. 

Look at freezing rain and freezing drizzle portion of icing 
grid. Knows freezing drizzle is occurring in western PA. 

May use freezing drizzle in wording of AIRMET. 
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Manually adds icing to join two separated areas. 
Manually deletes icing. 

Comments and Enhancements: 

Would like to see freezing level on cross-section. 
Would like to see relative humidity fields. 


Cross-section is useful to determine icing fields. Could 
also be use to determine cloud tops. 

Need location indicator on cross-section to show where 
cursor is in horizontal view. 

Skew-Ts not used but would be beneficial. 

Not much utility to Volume Viewer. 

Looping control is useful. 

Blue grid of no-icing areas is distracting. Would prefer to 
see only the icing grids. 

Need more information on PIREPs than currently displayed. 

Need satellite animation (looping of time sequence). 

Animation (looping of time steps) is important since AIRMET 
is 6 hour product with an extended outlook. 

Algorithm parameters not modified. Not comfortable with 
parameters. Would not use until was comfortable with 
understanding of the science and statistics. 

Need a better busy indicator. 

Need a check (perhaps a prompt) so will not add icing to 
levels below the freezing level. For instance, a dialog box 
prompting the user "are you sure you want to add/delete ice 
between these flight levels?" 

Would like to see "raw" data that goes into algorithm (e.g., 
relative humidity) rather than just the derived product. 

Wants to be able to apply the NCAR/RAP algorithm to more 
models. 
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Would like the 'reload list' feature to update 
automatically. 

To make an AIRMET: 

Point and click to draw outline. Would like to be able to 
edit lines in order to avoid short distances from location 
identifiers. 

Transfer points to word processor in similar manner as 
Convective SIGMET position. 

For a 6-hour valid AIRMET: 

Would like to step through with snapshots at 3-hour 
intervals. 

Another possibility, but not preferred, is to have a 6 hour 
composite. 
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Forecaster: 6 


Scenario 1: 

1. Description: BOS area with a valid time of 18Z (analysis 
time of 15Z). Current satellite images not available on editor. 

2. Procedures: 

Using editor cross-section, scan South to North and back. 
Checking heights of bases and tops. Scan East to West. 

Use N-AWIPS to look at SAOs (text product) in area of 
interest. Identifying precipitation in area. 

Look at AFOS graphics for indication of precipitation. Look 
at radar tops. 

Look at satellite image to identify convective activity. 

Look at relative hiamidity fields on N-AWIPS to determine 
moisture trends. Look at 700 mb level since icing displayed 
on editor was in that general level. 

Scenario 2: 

1. Description: SLC area with a valid time of 18Z. 

2. Procedures: 

Look at freezing level from Eta on N-AWIPS. 

Look at 700 mb fields (relative humidity, advection) on N- 
AWIPS. 

Scan editor cross-section. 

Based upon wind flow at 700 mb and satellite image, would 
add icing. Would use freezing level as base and cloud top 
information from skew-Ts to determine top of icing area. 

Comments and Enhancements: 

# 

Include heights of PIREPs. 

Include radar data underlayed on icing grids. 

Include freezing level on the editor. 
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Skew-T is too small. Need larger display, possibly a two 
headed system which allows full screen display. 

Skew-Ts should display modifications, for example, user 
specified changes to freezing level. 

Would be helpful to see skew-Ts from observations in 
addition to model derived ones. 

Need full screen display so detail is not lost. This is 
important with high resolution satellite images. 

Models other than RUC need to be on the editor so forecaster 
can use model of choice. 

Font size needs to be user configured. 

Animation is needed. Past 24 hours of icing grids to see 
how models have been handling. In addition, animation of 
PIREPs and AIRMETs is needed to determine how well models 
are performing. 

Include lightning data so convective areas will not be given 
an Icing AIRMET. 

Flight levels are preferred over isobaric levels. 

Volume viewer does not show utility. It is too easy to lose 
locations (perspective). 

Looping control is beneficial. 

Scanning is an excellent tool. 

Would like to set boundaries on cross-section scanning in 
order to go back and forth over an area of interest. 

Terrain representation on cross-section is useful. Allows 
identification of terrain induced icing. 

Would like to have a prompt identifying what levels icing 
will be added when manually editing grids. Perhaps a 
default based upon temperature criteria (0-20°C) so 
unrealistic flight levels will not have icing added. 

Would like to have a busy indicator that is more apparent to 
the user. 

Wants editor to make some intelligent assumptions. 
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To make an AIRMET: 

Point and click to identify outline. Automatic transfer of 
points and states to word processor. Bases and tops should 
also be transferred. 

For a 6-hour valid AIRMET: 

Use 3 hour increments. 

Changes in 3 hour increments should be sent to word^ 
processor for inclusion into AIRMET. For example/ if icing 
conditions end on a westward edge in the next 3 hour 
increment, then the change should be reflected in the 
information transferred to the word processor. 

Would want to animate (loop) edited grids in order to check 
for consistency and smoothness between the increments. 
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Forecaster: 7 


Scenario 1: 

1. Description: BOS area with a valid time of 21Z. 

2. Procedures: 

Look at RUC neural network icing product on N-AWIPS. Scan 
vertically through neural network levels and comparing to 
icing grid on editor. 

Based upon neural network/ identify 6-12 Kft as levels of 
icing. 

Using editor skew-Ts to define thickness of moist layers. 
Thin layers manually deleted. 

Manually deletes icing areas above 12 Kft based upon neural 
network guidance. 

Use cross-section to check bases and tops. 

Scenario 2: 

1. Description: SLC area with a valid time of 21Z. 

2. Procedures: 

Look at RUC neural network icing product on N-AWIPS. Scan 
vertically through neural network levels and comparing to 
icing grid on editor. 

Manually delete cloud free regions on editor. 

Manually add icing areas based upon neural network 
composite. 

Using N-AWIPS/ overlay PIREPs on satellite to identify 
significant areas of icing. 

Manually remove icing areas based upon neural network 
composite. 

Manually scan horizontal plan view and examining cross- 
section. 
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Comments and Enhancements 


Top levels of skew-T are not needed. Including this 
information degrades the detail and limits the size of. the 
skew-T, Use resolution as displayed on N-AWIPS which allows 
better cloud top estimates. 

Looping control is beneficial. However, manual stepping 
through the vertical levels is more important. 

Need a busy indicator. 

Voliime viewer is a "toy" that is not beneficial unless 
editing can be done in 3-d space. 

Editing on the cross-section would be useful, for example, 
sloping bases could be visualized. 

Icing intensities are needed. If in a contour product, it 
would be useful to edit the contours. 

Editing of the neural network icing product would be useful 
since the neural network product gives intensities. 

Would like to see PIREPs independent of grid time. 

Would help to stop skew t at 300 MB. Would enable you to 
increase skew t size without increasing window size 

As long as your fixed to a 2-D screen it is very difficult 
to truly utilize 3-D. 

Overall, editor is pretty good. 

Should be able to add intensities. 

Likes the cross-section. 

Didn't use algorithm parameter. Partly because unsure that 
he doesn't think it's the right algorithm. 

To make an AIRMET: 

Editor may make first guess for AIRMET outline but 
forecaster has to be able to modify if necessary. 

AIRMET outline should be automatically transferred to word 
processor. Short distances from VORs should be moved to the 
VOR. 



For a 6-hour valid AIRMET: 

Keep the time increments separate from one another. 

May want a composite of the time increments as a final 
check, but not as a final product. 

Want to keep temporal grids separate. Might want to do a 
composite at end to review all 3 times. Need to loop 
temporal data. 
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Forecaster: 8 


Scenario 1: 

1‘. Description: BOS area with valid time of OOZ. 

2. Procedures: 

Compare editor icing grids with neural network icing product 
on N-AWIPS. Vertically scanning neural network levels. 

Look at PIREPs on editor. 

Examining moisture profile of cloud regions. Using editor 
skew-T to check depth and vertical location of moist areas. 
Comparing to freezing level. 

Check Nested Grid Model (NGM) icing thickness on N-AWIPS. 
General agreement is seen with editor grids. 

Scenario 2: 

1. Description: SLC and SFO areas with a valid time of OOZ. 

2. Procedures: 

Using looping controls to determine tops and bases of icing 
areas. 

Check NGM icing thickness on N-AWIPS. Some differences .from 
editor grids identified. 

Change flight levels to only those of interest. Manually 
adds icing based upon what shown on NGM. 

Comments and Enhancements 

Cross-section is very useful, but would like to not be 
limited to only North-South and East-West. North-South 
cross-section would probably be used the most since 

variability in the bases is usually oriented north to south. 

0 

Cross-section needs geographical indicators. 

Skew-T is useful. 

Looping controls appear useful. However, cross-section is a 
better way for determining bases and tops. 
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In order to use the algorithm parameters, would have to use 
the icing product over time in order to see if any trends 
are identified (e.g., constant overforecasting). 

Would prefer a larger display. 

Addition of freezing level would be useful. 

The volume viewer does not appear to be useful. 

Valid time on the display needs to be more prominent. 

When looking at a single level, the level needs to be 
prominently displayed. 

Would like to see PIREPs independent of grid time. 

Prefers fixed menus that do not click on and off, such as 
with N-AWIPS. 

Would like to see additional information on the editor, such 
as relative hiunidity, temperature, and freezing level. 

Editor could be used to amend existing AIRMETs. 

Would need more than one monitor to view different models. 
Display would become too cluttered in limited to one 
display. 

Algorithm could be useful if you noticed trends that it was 
overforecasting. 

Doesn't see use for the volume viewer. 

Need data level indicator on loop as well as plan view. 

Would like additional indicator as to what level your at in 
looping control. ' 

Need more flexibility as to how you can view PIREPs — so 
would like to look at PIREPs for any time. 

Would like to see an automated product that developed 
AIRMETs within current constraints. 

Would like to see cross-section on an angle other than 
North/South or East/West. 

On North/South scan, you need some indication between Canada 
and US so you can see border. 
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To make an AIRMET: 


Point and click to outline area could be a possible option. 

Would be useful to have location identifiers to define 
points. 

Does not prefer an editor determined AIRMET area. The 
editor would probably generate an area with too many 
identifier points. The forecaster should be able to outline 
the area. 

Would like to create AIRMETsin the same fashion as the 
convective SIGMET VDUC. 

Would be useful to have VOR points. 

For a 6-hour valid AIRMET: 

Create a composite from 3 hour increments. Overlay 
beginning and ending times to form a total area. 

Accumulate in the overlay — create a single composite. 
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Forecaster: 9 


Scenario 1: 

1. Description: CHI area with a valid time of 03Z. 

2. Procedures: 

Zoomed in on area of interest. 

Looking at cross-section to identify vertical extent of 
icing layer (i.e., tops and bases). 

Look at skew-Ts in icing regions to determine vertical 
extent of moist areas. Compare with tops and bases from 
cross-section. 

Manually deleted icing areas. 

Scenario 2: 

1. Description: CHI area with a valid time of OOZ. 

2. Procedur.es: 

Comparing Eta icing output on N-AWIPS. 

Based upon knowledge of PIREPs, editor grids are realistic. 
Determine bases and tops from editor cross-section. 

Comments and Enhancements 

Would add satellite animation in order to know cloud trends 
and movements. Would include at least the past 6 hours of 
satellite images, maybe the last 12 hours. 

Looping Controls are useful. Manually stepping through 
levels is useful. 

The horizontal plan view scanning feature should include a 
step-by-step such as the Looping Controls have. This would 
allow the forecaster to manually step through cross- 
sections. 

The mouse was very sensitive. 

The horizontal plan view and the vertical cross-section need 
to be linked so when the horizontal view,is zoomed, the 
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cross-section automatically adjusts so that the two views 
are correlated geographically. 

Zooming is a useful function. Would not want pre-determined 
forecast area (FA) views since weather crosses FA 
boundaries. 

Wants to be able to loop the satellite imagery at least 6 
hours worth. 

Would be helpful to have some sort of marker in the vertical 
cross-section to let you know where you are. 

Loop controls are a useful tool — a step by step feature 
is good. 

System is not fast enough. 

Found it to be very tedious to have to keep the mouse 
perfectly still when moving scanning line. If user moved 
the mouse at all while the mouse button was depressed, it 
would start to draw a line. 

Nice zoom feature but too difficult to use. 

Would like options for zoom from pre-set to free zoom. 

Some sort of indication as to which Skew T is up on the 
chart. 

Font on Skew T is very difficult to read. 

Ensure that enlargement of Skew T doesn't obscure window. 

Need three hours of information either side of current time. 

Wants to be able to put PIREPs in 3D image. 

Loop the development of clouds — time loop how clouds are 
building or changing over 6 hours or 12 hours. 

To make an AIRMET: 

Point and click to outline an area. 

Overlay location identifiers. 
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For a 6-hour valid AIRMET: 


Edit each 3-hour increment during the AIRMET valid period. 
Manually step through each time increment (as current N- 
AWIPS capability). 



Forecaster: 10 


Scenario 1: 

1. Description: CHI area with a valid time of 18Z. Analysis 
time was 12Z. 

2. Procedures: 

Examine 12Z analysis grid on editor. 

Use Looping Controls to identify where the levels where the 
majority of the icing is. 

Change to 18Z forecast grid on editor. 

Use manual stepping of Looping Controls to scan levels. 

Check the four categories of the algorithm (i.e./ stable, 
unstable, freezing rain, and freezing drizzle). 

Varying the location of the cross-section in order to 
identify vertical and horizontal extent of forecasted icing 
area. 

Look at PIREPs on N-AWIPS (AFOS) to get details of reports. 
Looking at reports of both icing and no icing. Looking at 
vertical extent of PIREPs. 

Examine levels on editor that correspond to PIREPs of icing. 

Change to 15Z forecast. Compare with PIREPs. Appears that 
forecasted icing grids are lagging PIREPs of icing. 

Change back to 18Z forecast. 

Manually delete small portion of icing grid. 

Define flight levels and manually add icing based upon PIREP 
information. 

Switching between 15Z and 18Z forecasted grids to assess 
model performance. 

Scenario 2: 

1. Description: SLC and SFO area with a valid time of 18Z. 

2. Procedures: 
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Look at PIREPs on N-AWIPS (AFOS) and compare with 12Z 
analysis to assess model performance. 

Look at editor cross-section to determine icing levels.. 
Compare to PIREPs. 

Change to 15Z forecasted icing grid. 

Manually remove icing grid point outliers. Considered to be 
noise. 

Using cross-section to review icing in mountainous areas. 
Cross-section reveals areas to be thin layers and low level. 
Manually delete areas. 

Define flight levels and manually add icing to extend 
forecasted area. 

Comments and Enhancements 

Would like to have another window to compare the different 
categories of the icing algorithm. 

Include time animation. 

Vertical resolution is not used id full extent since AIRMET 
must provide an envelope of conditions. 

Cross-section is useful for giving the horizontal and 
vertical extent, but needs time animation. 

Cross-section needs to zoom to align with horizontal view 
when horizontal view is zoomed. 

More PIREP information is needed,, such as time and altitude. 
Shouldn't have to switch between editor and AFOS to get 
PIREP information. 

Would like to be able to control density of plotted PIREPs 
similar to AFOS feature. 

Grid covers satellite image. Get rid of grid where there is 
no icing. 

Have to be able to save edited grids. 

Volume Viewer has no utility for operations. 
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Skew-T was not used, but is probably useful for looking at 
specific features. However, the Skew-T is limited to a 
specific point rather than a broader horizontal extent. 

Must be familiar with algorithms before making any 
adjustments to algorithm parameters. 

Busy indicator needs to be more obvious. 

Include coastal water boundaries for forecaster's area of 
responsibility. 

Need user specified colors since some forecasters are color 
blind. 

Need forecasted information for the type of icing, not just 
whether it exists or not. 

Zoom and pan functions need to be easier. Should not have 
to use mouse in conjunction with keyboard. 

Stepped zoom as currently on editor is preferred. 

Freezing level would be a good addition as an overlay. 
Prefers color filled product rather than contours. 

Outliers should be manually removed. Manual editing is easy 
and ,there may be instances where outliers are representative 
of an actual icing threat. 

Would like to have another window available to see different 
regimes. 

Horizontal cross-section gives good detail for vertical 
extent. 

Current plan view should be looped in time along with the 
lower window. 

Need more detail in PIREPs: 

time 

type 

level 

severity 

Wants to be able to toggle on and off various PIREP 
information. 
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Would usually look at a couple of models to decide which is 
best one for the day. 

Really wants to turn blue grids off — don't need to see 
blue grids. 

Likes that vertical cross-section shows icing even when it's 
not in the selected range. 

Would really like to click on VORs. 

For PIREPs, would like both progressive disclosure and a 
data density control. 

Really need a freezing level overlay to be able to toggle on 
and off. 

To make an AII^MET: 

Point and click (similar to Convective SIGMET process). 
Editor should generate From Line and make adjustments to 
prevent small distances (10-20 miles) from location 
identifiers. 

Include location identifiers. 

Include a text window with editor so do not have to have 
external connection to a PC for generating AIRMET text. 

Outline horizontal area first, then the vertical levels 
second. 

For a 6-hour valid AIRMET: 

Would edit beginning time step and ending time step and then 
see composite. 



